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Abstract 


Enrollment in online courses has outpaced overall university enrollment for the past 
several years. The growth of online courses does not appear to be slowing. The purpose 
of this study was to compare course completion and student academic performance 
between online and traditional courses. Archival data from the host university student 
records system was collected using the Structured Query Language. Descriptive 
statistics were used to analyze student characteristics. Chi-square analysis was used to 
determine if statistically significant differences existed between students enrolled in 
online and traditional courses when comparing course completion and academic 
performance. Analysis found statistically significant differences existed in both course 
completion and academic performance for students enrolled in online versus traditional 
courses. Additional analysis indicated statistically significant differences existed in 
course completion by course discipline. 
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Introduction 


In the fall of 2007, more than 17.9 million students were enrolled in degree-granting, 
postsecondary institutions. Of those students, more than 3.9 million were enrolled in 
one or more online courses. Online enrollments represented 21.9% of total enrollments. 
From fall 2002 to fall 2007, online enrollments grew at a compound annual growth rate 
of 19.7% from 1.6 million to more than 3.9 million (Allen & Seaman, 2008). With the 
growth of online course enrollments, questions have been asked about course 
completion and student performance in online courses compared to traditional, face-to- 
face courses. 

Russel (2001) compiled an annotated bibliography of 355 research reports that 
examined differences in student outcomes between online and traditional courses. The 
majority of research revealed no statistically significant difference in student outcomes 
based on delivery mode. However, recent research on course completion and 
performance has been inconsistent. Brady (2001), Carr (2000), and Simpson (2003) 
found that course completion was generally lower in online courses when compared to 
traditional courses. Roach (2002) found that some institutions reported equal or higher 
course completion rates in online courses when compared to traditional courses. The 
problem this research will address is that further evidence is needed regarding student 
success measures in online instruction as opposed to classroom delivery. 

Growth in online course enrollments is clear and many researchers agree that the future 
of higher education is tied to some form of online course delivery (Berger & Lyon, 2005; 
Harasim, 2000; Palloff & Pratt, 2003). Are course completion rates significantly 
different between online and traditional courses? Is a student’s academic performance 
independent of the course delivery method? These questions are important to 
administrators tasked with maintaining the competitive and economic future of their 
respective universities and are the focus of this research. 

Formula Funding 

Texas public universities receive flexible, discretionary state funds based primarily on 
the formula funding calculation under the direction of the Texas Higher Education 
Coordinating Board (THECB). Of all the funds appropriated directly to Texas public 
universities, more than 62% comes from formula funding. The primary source of 
formula funding is generated by the instruction and operations formula. This formula is 
based on semester credit hours applied to a cost matrix identifying weights based on 
level of instruction (lower division, upper division, master, doctoral, or professional) 
and discipline (liberal arts, science, fine arts, etc.). The other formulas, Teaching 
Experience, Small Institution Supplement, and Infrastructure, also include semester 
credit hours, but total less than 25% of formula funding for most universities (THECB, 
2008). Current formula funding is based on the 12 th day of class enrollments and does 
not take into account students who drop courses during the semester. 
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In the state of Texas, course completion rates have become important with regards to 
funding. The THECB recommended a shift in formula funding for the 2010-2011 
biennium. The proposal would change the funding formula from using attempted to 
completed semester credit hours (THECB, 2008). With the continued growth of online 
education and the proposed shift in funding, course completion rates in online courses 
became increasingly important to administrators at Texas public universities. 

Another State of Texas rule for public university funding related to the number of times 
a student can repeat a course, commonly referred to as the “Three-Peat” rule (Texas 
Administrative Code [TAC], Title 19, Part 1, Rule 13.105, 2005). This rule has an impact 
on the student and university. The three-peat rule states that “Institutions shall not 
submit for formula funding any hours for a course that is the same or substantially 
similar to a course that the student previously attempted for two or more times at the 
same institution” ([TAC], Title 19, Part 1, Rule 13.105, 2005). In order for the university 
to compensate for the loss of funding, the student can be charged full out-of-state 
tuition cost for any course impacted by the three-peat rule. The three-peat rule 
increased the importance of student academic performance to avoid potential loss of 
state funding for the course and possible increased financial burden on the student. 

Course Completion 

Research on course completion rates between online and traditional course delivery has 
been mixed (Carr, 2000). Several studies showed differences existed in course 
completion rates between online courses and traditional, face-to-face courses (McLaren, 
2004; Paden, 2006; Roach, 2002). Waschull (2001) found that online course 
completion rates were not significantly different from traditional course completion 
rates. Nelson (2006) compared course completion rates between online and traditional 
courses at Delaware Technical and Community College, Terry Campus. Her research 
found statistically significant differences in the course completion rates between online 
and traditional courses. Additional analysis showed that more students (23%) withdrew 
from the online courses compared to the withdrawal rate in traditional courses (18.4%). 

Carr (2000) noted that some universities reported drop-out rates as high as 80% in 
online courses, but argued that course completion rates should not be compared across 
universities since universities report course completion rates differently. Some 
universities included students who dropped during the add/drop period while other 
universities did not report those instances. Without having a standard rule, a 
comparison across universities might not produce accurate results. Regardless of the 
ability to compare across universities, research into course completion rates is useful to 
university administrators tasked with determining class size and number of sections as 
well as those administrators responsible for assessing student learning outcomes for 
internal and external agencies (McLaren, 2004). 


Vol 14 | No 4 


Oct/13 


106 





Comparison of Course Completion and Student Performances through Online and Traditional Courses 


Atchley, Wingenbach, and Akers 


Student Performance 

Russell (2001) explored 355 research reports comparing student outcomes between 
different course delivery modes. The majority of the research indicated no statistically 
significant differences existed in student outcomes based on delivery mode. Clark 
(1994) stated that it was the teaching methods and not the delivery medium used that 
influenced learning. In all cases, Clark argued that the selection of course delivery mode 
should be an economic decision (1994). 

Much research exists supporting Russell’s (2001) work (Clark, 1994; Gagne & Shepherd, 
2001; McLaren, 2004). However, several studies have found statistically significant 
differences in student outcomes based on delivery type (Faux & Black-Hughes, 2000; 
Paden, 2006; Shoenfeld-Tacher, McConnel, & Graham, 2001). Paden found a 
statistically significant difference in student performance between online and traditional 
courses. Faux and Black-Hughes conducted research into student performance between 
different delivery modes of a social work course and found that a statistically significant 
difference existed between post-test scores by delivery mode. Additional analysis 
indicated that students in the online section did not perform as well as students in the 
traditional section. 

Course Completion by Discipline 

Several researchers found that certain disciplines were not suited for an online setting 
(Carnevale, 2003; Nelson, 2006; Noble, 2004; Paden, 2006; Smith, Heindel, & Torres- 
Ayala, 2008). Lab science, health care (Carnevale, 2003), and mathematics (Smith et 
al., 2008) courses have all been identified as course disciplines that are not well-suited 
for online course delivery. Terry (2001) suggested that courses such as accounting, 
economics, computer information systems, marketing, and management were 
potentially more conducive to online course delivery. 

Smith et al. (2008) compared attrition rates between mathematics-related courses and 
non mathematics-related courses and found higher attrition rates in the mathematics- 
related courses. Attrition in traditional mathematics-related courses was the same as 
the drop-out rates in non mathematics-related courses. The researchers suggested that 
“for online students, mathematics is not working as well as other disciplines online, and 
further, students' difficulties with mathematics relative to other disciplines are not as 
great as in the face-to-face modality” (p. 152). 

Online course delivery has been a growing area in higher education (Allen & Seaman, 
2008). As universities continue to transition courses to online delivery, it is important 
to understand the impact on student performance and retention. Is student 
performance in online courses comparable to performance in traditional courses? Are 
there differences in course completion in online courses when compared to traditional 
courses? Are some disciplines more appropriate for online delivery? 
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Purpose and Objectives 

The purpose of the study was to compare course completion rates and student 
performance between online and traditional courses. The following research objectives 
were used to guide the study: 

1. Determine if there was a statistically significant difference in the 
performance of students enrolled in online courses compared to students 
enrolled in traditional courses. 

2. Determine if there was a statistically significant difference in the retention 
of students enrolled in online courses compared to students enrolled in 
traditional courses. 

3. Determine if there was a statistically significant difference in the retention 
of students enrolled in online courses by course discipline. 


Methodology 


This study had a causal-comparative research design, using archival data for students 
enrolled at a small, regional, public, comprehensive university located in the southwest 
United States. Archival data was used to compare both course completion and student 
academic performance (measured by final course grade) between groups of students in 
online and traditional courses. This same approach was used to compare course 
completion of students enrolled in online courses by course discipline. 

Causal-comparative designs do not allow for explicit finding of causation (Fraenkel & 
Wallen, 2006), but do strongly suggest whether mode of instruction had a direct impact 
on student retention. Additionally, since causal-comparative design takes place after 
data were collected and without any manipulation or intervention, it allowed for the 
exploration of naturally occurring relationships between groups. 

The population (N = 319,153) for this research was student course experiences for 
students enrolled in all 16-week courses taught between fall 2004 and spring 2009. 
Summer semesters were excluded as they are not comparable in structure or design to a 
fall or spring course (Nelson, 2006). Online courses as defined by Allen and Seaman 
(2008) were those courses “in which at least 80 percent of the course content is 
delivered online” (p. 4). For the purpose of this research, online courses were defined as 
those courses designated in the target university’s student records system with an online 
building designation. Traditional courses were defined by Allen and Seaman (2008) as 
courses “with no online technology used - content is delivered in writing or orally” (p. 
4). For the purpose of this research, traditional courses were defined as those courses 
designated in the student records system with a building code other than online and 
with an instruction code of “lecture.” 
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For objective one, a purposive sample was selected that consisted of all student course 
experiences for students who were enrolled in and completed courses taught in both an 
online and traditional lecture format by the same professor during the same semester 
and year (n = 5,477). The total number of distinct students enrolled in the courses from 
the sample was 4,120. 

For objective two, a purposive sample was selected that consisted of all student course 
experiences for students enrolled in courses taught in both an online and traditional 
lecture format by the same professor during the same semester and year (n = 5,778). 
The total number of distinct students enrolled in the courses from the sample was 
4 , 307 . 

For objective three, the sample included all student course experiences for students 
enrolled in online courses taught consistently during each of the 16-week semesters 
between fall 2004 and spring 2009 (n = 8,445). Of the 41 subject areas with courses 
delivered between fall 2004 and spring 2009, only 14 subject areas were delivered 
during each of the semesters between fall 2004 and spring 2009. The total number of 
distinct students enrolled in courses from the sample was 3,932. 

Data for the research objectives were extracted from the student records system using 
the Oracle Structured Query Language (SQL). For objectives one and two, all courses 
taught during full 16-week semesters were identified and extracted into a temporary 
table. Using the temporary table as the research sample, a SQL statement was written 
to find all courses with both online and traditional sections taught by the same professor 
during the same semester and year. With the sample courses identified, course delivery 
mode, registration status, and final grade were extracted for all students enrolled in the 
sample courses. No student identifying information was included in the data extraction. 

For objective one, a comparison of student academic performance, all students with a 
registration status code other than dropped or withdrawn were included in the sample. 
This objective was intended to measure academic performance, only students who 
completed the course were included. Student academic performance was measured by 
the final course grade. 

For objective two, a comparison of course completion, all students enrolled in the 
sample courses were used. Students with a grade code of A, B, C, D, or F and 
registration status other than dropped or withdrawn were flagged as retained and all 
others were flagged as not retained. 

For objective three, a comparison of online course completion by course discipline, data 
extracted included registration status, final grade, and course subject for all students 
enrolled in all online courses identified in the initial base course table. Course discipline 
was defined as the subject area a course was associated with for a particular college and 
department. Subject areas were designated by a four character abbreviation and 
assigned to all courses listed in the host institution’s course catalog. 
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Pearson’s chi-square test was used to determine if statistically significant differences 
existed in retention or performance by mode of instruction. “Chi-square is the 
inferential technique used to determine statistical significance of a relationship” (Wallen 
& Fraenkel, 2001, p. 530). An alpha level of .05 was set a priori to determine if an 
association existed between the independent variable - delivery mode - and the 
dependent variables retention and performance. For objective three, a chi-square test 
was calculated on all students enrolled in online courses to determine if there was a 
statistically significant difference in the expected and observed retention by course 
discipline. 


Results 


Objective one was to determine if a statistically significant difference existed in 
academic performance of students enrolled in online courses compared to students 
enrolled in traditional courses. A chi-square analysis was performed in SPSS to 
determine if there was a statistically significant difference in student performance 
between students enrolled in online courses and students enrolled in traditional 
courses. Students enrolled in online courses had the highest percentage of As at 34.6% 
compared to students enrolled in traditional courses at 31.3%. Table 1 displays the 
grade distribution by course delivery modality. The difference in performance was 
statistically significant, x 2 (4, N = 5,477) = 27.383, pc. 05 (see Table 1). 

Table 1 

Contingency Table for Students’ Academic Performance (N = 5,744) 


Modality 

A 

( 96 ) 

B 

( 96 ) 

C 

( 96 ) 

D 

( 96 ) 

F 

( 96 ) 

Total 

Online 

631 

562 

296 

141 

195 

1,825 


34 - 6 % 

30.8% 

16.2% 

7.7% 

10.7% 


Traditional 1,142 

1,180 

762 

270 

298 

3,652 


31 - 3 % 

32.3% 

20.9% 

7.4% 

8.2% 


Total 

1,773 

1,742 

1,058 

411 

493 

5,477 


32.4% 

31.8% 

19.3% 

7 - 5 % 

9.0% 


Note. x 2 = 

27.383, critical value = 

9.49, d.f. 

= 4 





Objective two was to determine if there was a statistically significant difference in the 
retention of students enrolled in online courses compared to students enrolled in 
traditional courses. A chi-square analysis was performed in SPSS to determine if there 
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was a statistically significant difference in course completion between students enrolled 
in online courses and students enrolled in traditional courses. Students enrolled in 
online courses had the lowest course completion rates at 93.3% compared to students 
enrolled in traditional courses at 95.6%. Table 2 displays the course completion 
distribution by course delivery modality. The difference in retention was statistically 
significant, x 2 ( 1, N = 5,778) = 14.132, p < .05 (see Table 2). 

Table 2 

Contingency Table for Course Completion Rates (N = 5,778) 


Modality 

Retained 

( 96 ) 

Not retained 

( 96 ) 

Total 

Online 

1,825 

132 

1,957 


93 - 3 % 

6.7% 


Traditional 3,652 

169 

3,821 


95 - 6 % 

4.4% 


Totals 

5,477 

301 

5,778 


94.8% 

5.2% 


Note, x 2 = 

14.132, critical value = 5.99, d.f. 

= 2 



Objective three was to determine if there was a statistically significant difference in the 
retention of students enrolled in online courses by course discipline. A chi-square 
analysis was performed in SPSS to determine if there was a statistically significant 
difference in students’ course completion by course discipline. The 14 different 
disciplines included in the chi-square analysis were accounting, agricultural education, 
computer information systems, English, finance, general business, health, human 
resource management, management, marketing, physical education, psychology, 
reading, and special education. Finance had the lowest course completion rate at 82.2% 
compared to reading with the highest retention rate at 98.2%. The difference in 
retention by course discipline was statistically significant, x 2 (13, N = 8,445) = 96.974, p 
< .05 (see Table 3). 
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Table 3 

Contingency Table for Course Completion by Course Discipline (N = 8,445) 


Modality 

Retained 

Not retained 

% Retained 

Total 

Computer information systems 

i, 3 n 

118 

91.7 

1,429 

General business 

1,187 

88 

93 -i 

1,275 

Management 

1,000 

81 

92.5 

1,081 

Psychology 

924 

93 

90.9 

1,017 

English 

654 

63 

91.2 

717 

Physical education 

450 

23 

95-1 

473 

Marketing 

417 

33 

92.7 

450 

Special education 

386 

16 

96.0 

402 

Health 

339 

11 

96.9 

350 

Human resource management 

278 

40 

87.4 

318 

Finance 

245 

53 

82.2 

298 

Agricultural education 

271 

22 

92.5 

293 

Accounting 

151 

24 

86.3 

175 

Reading 

164 

3 

98.2 

167 

Totals 

7,777 

668 

92.1 

8,445 


Note, x 2 = 96.974, critical value = 22.36, d.f. = 13 


Conclusions and Recommendations 


The growth rate of student enrollments in online courses is outpacing the growth rate of 
the total higher education student population (Allen & Seaman, 2008). Research on the 
course completion rates in online education is mixed. Some research has found course 
completion in online courses was as good as or better than in traditional courses 
(Roach, 2002). Other researchers have found that traditional courses have higher 
course completion rates when compared to online equivalents (Brady, 2001; Carr, 
2000; Simpson, 2003). 

Course completion and student performance has financial impacts on students as well 
as the university. The THECB (2008) proposed a shift in formula funding from 
attempted to completed semester credit hours. If retention and completion in online 
courses is lower than the traditional classroom setting, the host university could 
potentially lose a portion of state funding. Understanding where retention in online 
courses is a problem will allow the host university to take corrective action in order to 
increase retention and student success in online courses. Additionally, the Texas 
Administrative Code dictates that a university “shall not submit for formula funding any 
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hours for a course that is the same or substantially similar to a course that the student 
previously attempted for two or more times at the same institution” ([TAC], Title 19, 
Part 1, Rule 13.105, 2005). To compensate for the loss of state funding, the university 
could charge tuition up to the out-of-state tuition rates for the course. 

Objective one sought to determine if statistically significant differences existed in 
student performance between online and traditional courses. A chi-square analysis on 
the dataset indicated that a statistically significant difference did exist in the student 
performance between online and traditional courses. This finding supports previous 
research on student performance in online courses (Faux & Black-Hughes, 2000; 
Paden, 2006; Shoenfeld-Tacher, McConnel, & Graham, 2001). Additional observation 
of the grade frequencies found a higher percentage of As, Ds, and Fs in online courses, 
while traditional courses had a higher percentage of Bs and Cs. Shoenfeld-Tacher et al. 
found student academic performance as measured by a post-test in an online science 
course was significantly different and superior to student performance in the traditional 
course section. Paden found that student academic performance in an introductory 
math course was significantly different between online and traditional delivery. 
Contrary to what Shoenfeld-Tacher et al. found, Paden noted academic performance of 
students enrolled in the traditional section of the introductory math course was superior 
to students enrolled in the online section. 

With regard to objective two, statistically significant differences did exist in course 
completion rates between online and traditional course delivery. This finding supports 
research conducted by McLaren (2004), Paden (2006), and Roach (2002) who found 
differences in course completion rates between online and traditional courses. 
Additional analysis indicated that students enrolled in online courses had a lower course 
completion rate (93.3%) than students enrolled in traditional courses (95.6%). This 
supports research by Paden who found that traditional course delivery had a higher 
retention rate compared to online delivery for students enrolled in an introductory math 
course. Nelson (2006) found statistically significant differences in student retention 
rates between online and traditional course delivery. 

With regard to objective three, statistically significant differences existed in course 
completion rates by course discipline. Additional observations supported previous 
research that suggested some disciplines may not be well-suited to online delivery 
(Carnevale, 2003; Nelson, 2006; Noble, 2004; Paden, 2006; Smith, Heindel, & Torres- 
Ayala, 2008). Course completion varied by discipline with reading having the highest 
rate at 98.2% and finance with the lowest at 82.2%. Nelson examined course 
completion rates for nine disciplines and found that no statistically significant 
differences existed for seven of the disciplines. However, statistically significant 
differences did exist in criminal justice and psychology and Nelson suggested that these 
course disciplines might not be conducive to online delivery. Smith et al. (2008) 
compared online and traditional course completion rates in mathematics courses and 
found lower retention rates in online mathematics courses. The researchers suggested 
that mathematics might not be appropriate for online delivery. 
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The research was conducted using archival data from the host institution’s student 
record system. No data was available on student perceptions of the courses or student 
aptitude with the technology used for course delivery. Additional student 
characteristics such as age, gender, ethnicity, classification, major, and experience with 
online course delivery were not evaluated as part of this research. What type of student 
is likely to succeed in online courses? Does experience with the technology lead to 
greater course completion and improved student performance in online courses? More 
research into student characteristics could help identify possible variables to predict 
student success in online courses. 
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